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Abstract. In this paper, we are interested in the almost sure convergence of randomly 
truncated stochastic algorithms. In their pioneer work, Chen and Zhu (1986) required that 
the family of the noise terms is summable to ensure the convergence. In our paper, we present 
a new convergence theorem which extends the already known results by making vanish this 
condition on the noise terms — a condition which is quite hard to check in practice. The aim 
of this work is to prove an almost sure convergence result of randomly truncated stochastic 
algorithms under easily verifiable conditions (see Theorem 1). 
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1. Introduction 

The localisation of the zeros of a function u is a quite complicated problem for which 
many techniques have already been developed. The use of stochastic algorithms is widely 
spread for solving such problems. Stochastic algorithms are particularly well suited where 
some on-line parameter estimation is needed. Such algorithms go back to the pioneer work 
of Robbins and Monro (1951). They proposed to consider the following recurrence relation 

X n+ i = X n - j n+ iu(X n ) - j n+ i5M n+ i, 

where 7 n is a decreasing gain sequence and Am n the measurement error. Under certain 
conditions on the growth of the L 2 — norm of the error, X n converges almost surely to the 
unique root of u. Since their work, much attention has been drawn to the study of the theory 
of such recursive approximations. The first works were dealing with independent measurement 
error on the observations. A great effort was made in this direction to weaken the conditions 
imposed on both the regressive function and the noise term. Using the ordinary differential 
equation technique, Kushner and Clark (1978) proved a convergence result for a wider range 
of measurement noises and in particular for martingale increments. 

One major drawback of these algorithms is that their convergence can only be established 
if the function u does not grow too quickly, namely a sub-linear behaviour is required. This 
is a dramatic restriction for practical applications. Chen and Zhu (1986) have found a way to 
get round the restriction by considering stochastic algorithms truncated at randomly varying 
bounds. Their algorithm can be written 

X n+1 = X n - ^ n+ iu(X n ) - 7„ + i(5M n+ i - 7„ + ijp n+ i, (1) 

where p n is a truncation term. 

In this paper, we are concerned with the convergence of the truncated algorithm (1). Several 
results already exist but the hypotheses considered differ quite significantly. The first result 
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concerning the almost sure convergence is due to Chen and Zhu (1986). The convergence 
was also studied by Delyon (1996) and Delyon et al. (1999). The robustness of the procedure 
was established by Chen et al. (1988) under global hypotheses on the measurement error. 
Namely, they require that the series ^ n j n 5M n converges almost surely. Delyon (1996) has 
also studied the almost sure convergence under local hypotheses on the measurement noise. 
Here, we give a self-contained proof of the convergence under local hypotheses, that is we 
only assume that ^ n "y n SM n l^ Xn ^K} converges almost surely for any compact set K. We do 
not impose any condition on the truncation term p n . 

First, we define the general framework and explain the algorithm developed by 
Chen and Zhu (1986). Our main result is stated in Theorem 2 in a very general way. For 
practical purposes, we give in Theorem 1 an easily verifiable condition under which our main 
result holds. This theorem is extremely valuable and dramatically extends the range of ap- 
plications of randomly truncated stochastic algorithms. Finally, Section 4 is devoted to the 
proof of the general convergence theorem. 



2. General framework 

Let us consider a general problem consisting in finding the root of a continuous function 
u: X G M d i — > u(X) £ M. d , defined as an expectation on a probability space (p,,A, P). 

u(X)=E(U(X,Z)), (2) 

where Z is a random variable in W 71 and U a measurable function defined on M rf x W 71 into M. d . 
We assume that x — > E(||?7(x, Z)|| 2 ) grow faster that ||a:|| 2 , so that the convergence of the 
standard Robbins Monro algorithm is not guarantied. Instead, we consider the alternative 
procedure proposed by Chen and Zhu (1986), on which we concentrate in this work. 

The technique consists in forcing the algorithm to remain in an increasing sequence of 
compact sets. Somehow, it prevents the algorithm from blowing up during the "first" steps. 

We consider an increasing sequence of compact sets (tCj )■ 

oo 

|J fCj = R d and Vj, fCj £ int(/C,+i) . (3) 

j=0 

We also introduce (Z n ) n an independent and identically distributed sequence of random vari- 
ables following the law of Z and (7n) n a decreasing sequence of positive real numbers. j n 
is often called the gain sequence. For any deterministic Xq £ /Co and <to = 0, we define the 
sequences of random variables (X n ) n and {o~ n ) n - 

X n — ^ n+ \U{X n , Z n+ i), 

X n+ i and a n+ i = a n , (4) 
X and a n+1 = a n + 1. 

Remark 1. When X n+ i ^ 1C an , one can set X n+ \ to any measurable function o/(Xq, . . . , X n ) 
with values in a given compact set. This existence of such a compact set is definitely essential 
to prove the a.s. convergence of {X n ) n . 

Remark 2. X.i represents the iterate of the Robbins Monro algorithm at step n + 1. 
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We introduce T n = u{Zk \ k < n) the a- field generated by the random vectors Zk, for k < n. 
Note that X n is T n — measurable since Xq is deterministic and U measurable. We can write 
u(X n )=E[U(X n ,Z n+1 )\F n }. 

It is often more convenient to rewrite (4) as follows 

X n+ i = X n - -y n+1 u(X n ) - j n+1 5M n+ i + j n+ ip n+1 (5) 

where 

SM n+1 = U(X n ,Z n+1 )-u(X n ), (6) 

ju(X n )+SM n+1 + ^-(X -X n ) i£X n+1 ^}C an , 
and p n+ i = < ln+1 ^2 (7) 

10 otherwise. 

Remark 3. 5M n is a martingale increment. The case of the standard Robbins Monro algo- 
rithm corresponds to p n = 0. 

3. Almost sure convergence 

In this section, we present a new convergence theorem that improves the result of 
Chen and Zhu (1986) who proved the almost sure convergence under global hypotheses on the 
series En j n +i5M n+ i whereas we can manage the proof under local hypotheses only, namely 
we only assume that the function x i — > E(||[7(x, Z)\\ 2 ) is bounded on all compact sets. Such 
a local hypothesis is much easier to satisfy in practical applications. 

Theorem 1. We assume that 

(Al) There exists a unique x* s.t. u(x*) = and Vx ^ x*, (u(x)\(x — x*)) > 0. 
(A2) En 7n = oo and E„ 7n < °°- 

(A3) The function x i — ► E(||C r (x, Z)\\ ) is bounded on any compact sets. 

Then, the sequence (X n ) converges a.s. to x* for any sequence of compact sets satisfying (3) 
and moreover the sequence (o~ n ) is a.s. finite (i.e. for n large enough p n = a.s.). 

We will not prove Theorem 1 directly as it actually derives from a more general result. 

Theorem 2. Under Hypothesis (Al) and if 

(M) En7n = 00. 

(A5) For all q > 0, the series ^2 n 1n+i^M n+ it^x n -x*\\<q} converges almost surely. 

Then, the sequence (X n ) converges a.s. to x* and moreover the sequence (o~ n ) is a.s. finite 

(i.e. for n large enough p n = a.s.). 

Remark 4. In the case where u derives from a potential V (i.e. u = W), Hypothesis (Al) 
is satisfied as soon as V is strictly convex. 

Proof of Theorem 1. It is sufficient to prove that the hypotheses of Theorem 1 imply the 
ones of Theorem 2. Consider M n = Ei=i 1i^-^i^-{\\Xi^i— x*\\<q}> 

(M n ) n is a martingale. By 

computing its angle bracket, we find (M) n = ^17=1 7 J 2 IE((5Mj5M J / |^ r j_i)l|||j 4 - i _ 1 - x -*||<g}- As 
the series E« if converges and the function x i — ► E(||C/(x, Z)|| 2 ) is bounded on all compact 
sets, the almost sure convergence of {M) n ensues from the Strong Law for square integrable 
martingales. Hence, we can apply Theorem 2, and the conclusion yields. ■ 
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4. Proof of Theorem 2 

The proof of Theorem 2 is based on the following lemma which establishes a condition for 
the sequence (X n ) n to be a.s. compact. 

Lemma 1. If for all q > 0, the series '^2 n>0 'y n 5M n t{\\x n - 1 -x*\\<q} converges a.s. and if 
p n l{||x n _ 1 -x*||<g} — y 0, then the sequence (X n ) n remains a.s. in a compact set. 

Note that the compact set mentioned in Lemma 1 is random. In particular, this lemma 
does not imply that the number of truncations is bounded independently of the randomness uj. 

Proof of Theorem 2. The proof is divided in two parts. 

• Let q > 0. We define M n = £)" =1 7i5M i l{\\x i - 1 -x*\\<q}- Thanks to Hypothesis (.45), 
M n converges almost surely. 

Assume that a n — > oo. This is in contradiction with the conclusion of Lemma 1, 
which implies that the hypothesis according to which p n ^-{\\x n - 1 -x*\\<q} tends to 
does not hold. So, 

3 77 > 0, q>0, \/N>0,3n>N ± { \\ Xn -x*\\< q } \\Pn+l\\ > V- 
Let e > 0. There exists a subsequence X^,r n \ such that for all n > 0, 

1 {\\X c , {n) -x*\\<q} ||lty(n)+l|| / and ||70(n)+l^(n)+l|| < £■ 

So, ||^(n) — 1 1 < Q an d however the new potential iterate X^ n ^ + i = X^^ — 

l4>(n)+l( u ( X <l>(n))+$ M <l>{n)+l) is not in ^ { „) ■ Since u is continuous, ||7<^(n)+l u (^(n))|| 
can be made smaller than e. As ||7^( n )+i<5^>(n)+i|| < e > a proper choice of e enables 
to write 

\\ X <t>(n) -x*- l^+AuiX^n)) + SM^ n)+1 )\\ <q+l. 

Let I be the smallest integer s.t. B(x*,q + 1) C /Q (such an integer exists thanks to 
(3)), then a^r n ) < I for all n. Since the sequence (<r n ) n is increasing, this proves that 
limsup n o n < oo a.s.. 

• According to the previous item limsup n a n < oo a.s.. So, the sequence (X n ) n is almost 
surely compact. Consequently, we can in fact set q = oo in Hypothesis (-45) and say 
that Yli JiSMi converges almost surely. Let us consider 

oo 

X' n = X n - ^ SM i- 

i=n+l 

Since the series ^2 i>0 7i<5Mj converges a.s. and X n remains in a compact set, X' n also 
remains in a compact set. Let C be this compact set. We define u = sup^gc ||tt(a;)||. 

X' n+1 = X' n - j n+1 u(X' n ) + 7„+ie„, 
where e n = u(X' n ) — u(X n ). Since \\X' n — X n \\ — ► and u is continuous, ||e n || — > 0. 

\\X' n+1 - x*\\ 2 < || A; - x*|| 2 -2 7n+1 (A; - x* I u(X' n )) 

+ 7n+l(4 + U 2 ) - 2 7 n+l(A' - X* I E n ). 

We can rewrite the inequality introducing a new sequence e' n — ► 0. 

||A; +1 -xi 2 < \\X' n -x*\\ 2 -2 ln+1 (X' n -x*\u(X' n ))+ ln+1 e' n . (8) 
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Let 5 > 0. If ||X 



x 



> 5, then (X^ — x* , u(X' n )) > c > 0. Henceforth, for n 



large enough Equation (8) becomes 



x 



< 



\X' n -x* 



-7 n+ iclr,i X / _ x *i|2 >5 ) + 7n+i(e + 4) 1 



{||x;-**|| a <«}' 

2 



{\\x> n -x*r>s} 

where c = sup|| a ,_ a ,*||2 <5 (x — x*\u(x)). Since X^n^'« = 00 > eacn time ||X^ — > 5, 

the sequence X' n is driven back into the ball B(x*, \/S) in a finite number of steps. 
Hence, for any n large enough 

2 



X 



< <5 + 7*(n)+l(c + 4( n )), 



n 



where 



limsup n | IX' 



sup{p < n; |X' 



< 5}. As 0(n) a.s. tends to infinity with n, 



< 5 for all 5 > 0. This proves that X' n — > x*. Finally, since the 



series ^ n 7 n+ i<5M n+ i converges, this also proves that X n 



x . 



Now, we are going to prove Lemma 1. 

Proof of Lemma 1. If a n < oo a.s., the conclusion of the Lemma is obvious. Assume that 
a n — ► oo. Since each time a n increases, the sequence X n is reset to a fixed point of /Co, the 
existence of a compact set in which the sequence lies infinitely often is straightforward. 

Let M > 0, we set C = {x; \\x — x*\\ 2 < M}. We can rewrite the Hypotheses of the Lemma 
as follows 



Ve > 0, 3 N > s.t. Vn,p> N we have < 



v 

T**Affc 1 {||X fc _ 1 -x*|| a <Jlf+2} 

k=n 

In < e, 

1 {||X„_ 1 - a; *|| 2 <A/+2} \\Pn\\ < e - 

Let e > and N > satisfying Condition (9) and s.t. Xn £ C. We introduce 



< e, 



0) 



X' — Xn 



E 

i=n+l 



7i^l{||X I _ 1 -x*|| 2 <M+2}- 



By using Equation (5), we can easily show that X' n satisfies the following recurrence relation 

X' n+1 =X' n - 7n+l£Wra+ll{||x„-x*|| 2 >M+2} ~ Jn+l(u(X n ) -p n +l)- (10) 

We will now prove that the sequence (X' n ) n remains in the set {x; \\x — x*\\ 2 < M + l} = C . 

The recurrent hypothesis is satisfied for n = N (it is sufficient to choose e < 1). Assume 
that the hypothesis holds for N, . . . ,n. Hence, ||X n — x*|| < M + 2. Then, we can deduce 
from Equation (10) that 

X' n+l = X' n - -y n+ i(u(X n ) - p n+ i), 

||X; +1 -^|| 2 < \\X' n -x*\\ 2 -2 7n+1 (X; - x* | u(X n )) + 7n+ice, 

where c is a positive constant independent of M. 



if WXL 



x 



< M, thanks to the continuity of u, a proper choice of e ensures that 



7n+i || (X; - x* I u(X n ))\\ < 1. Hence, ||x; +1 



< M + l. 
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• If M < \\X' n — x*\\ 2 < M + 1, thanks to the continuity of u and thanks to Hypothe- 
sis («41), (X n — x* | u(X n )) > S > 0. Once again, properly choosing e guaranties that 

ii 1 1 2 

ce < 5. Consequently, ||A^ +1 — x*\\ < M + 1. 

We have proved that for all n > N, \\X' n — x*|| 2 < M + 1. Since e can be chosen smaller 
than 1, the following upper-bound also holds 

\\X n - x*\\ 2 < M + 2, for all n > N. 

This achieves to prove that the sequence (X n ) n remains in a compact set and consequently 
that limsup n <7 n is a.s. finite. ■ 

Acknowledgement. I would like to thank Bernard Lapeyre for the fruitful remarks he made 
on a previous version of the proof presented above. 
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